Skip to content

ci: only check new links on pull requests to avoid rate limiting#5162

Merged
xrmx merged 1 commit intoopen-telemetry:mainfrom
MikeGoldsmith:mike/ci-check-links-diff-only
Apr 30, 2026
Merged

ci: only check new links on pull requests to avoid rate limiting#5162
xrmx merged 1 commit intoopen-telemetry:mainfrom
MikeGoldsmith:mike/ci-check-links-diff-only

Conversation

@MikeGoldsmith
Copy link
Copy Markdown
Member

@MikeGoldsmith MikeGoldsmith commented Apr 29, 2026

Description

The check-links CI job checks all links in changed markdown files. For files like CHANGELOG.md that contain hundreds of URLs, this triggers HTTP 502 rate limits from GitHub, causing spurious CI failures across many PRs.

Changes

  • Pull requests: only URLs from added lines in the diff are extracted and checked. This avoids re-checking hundreds of existing links that haven't changed.
  • Push to main: unchanged — all links in changed files are still checked for full validation.
  • Added fetch-depth: 0 to the checkout step so git diff has access to the base branch.

How it works

git diff origin/$base...HEAD -- <changed files>
  | grep '^+'           # only added lines
  | grep -v '^+++'      # skip diff headers
  | grep -oP 'https?:…' # extract URLs
  | sort -u             # deduplicate

The extracted URLs are written to a temporary markdown file and passed to markdown-link-check.

cc @emdneto

Large files like CHANGELOG.md contain hundreds of links. Checking
all of them on every PR triggers HTTP 502 rate limits. On PRs, now
only URLs from added lines in the diff are checked. Pushes to main
still check all links in changed files for full validation.

Assisted-by: Claude Opus 4.6
@MikeGoldsmith MikeGoldsmith requested a review from a team as a code owner April 29, 2026 16:31
@MikeGoldsmith MikeGoldsmith added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Apr 29, 2026
@MikeGoldsmith MikeGoldsmith moved this to Easy to review / merge / close in Python PR digest Apr 29, 2026
@github-project-automation github-project-automation Bot moved this from Easy to review / merge / close to Approved PRs in Python PR digest Apr 29, 2026
Copy link
Copy Markdown
Member

@emdneto emdneto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. Can you update CHANGELOG.md so we can verify the run?

@xrmx
Copy link
Copy Markdown
Contributor

xrmx commented Apr 30, 2026

Will this handle the case where some external links we have already in the codebase go 404?

@xrmx
Copy link
Copy Markdown
Contributor

xrmx commented Apr 30, 2026

Will this handle the case where some external links we have already in the codebase go 404?

Ah this is only for pull requests, sorry for the noise.

@xrmx xrmx merged commit 73678be into open-telemetry:main Apr 30, 2026
466 of 467 checks passed
@github-project-automation github-project-automation Bot moved this from Approved PRs to Done in Python PR digest Apr 30, 2026
@xrmx
Copy link
Copy Markdown
Contributor

xrmx commented Apr 30, 2026

Ah missed this was flagged with skip changelog, we'll see if this work on the next PR, sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants